68" International Astronautical Congress 2017, Adelaide, Australia. 


ITAC-17.A6.3.6x41656 


CHARACTERIZING DEBRISAT FRAGMENTS: SO MANY FRAGMENTS, SO MUCH DATA, AND SO 
LITTLE TIME 


B. Shiotani, M. Rivero, M. Carrasquilla, S. Allen! 
N. Fitz-Coy', J.-C. Liou”, T. Huynh’, M. Sorge*, H. Cowardin*, J. Opiela®, and P. Krisko® 


'The University of Florida, USA, ?National Aeronautics and Space Administration, USA, *United States Air 
Force/Space and Missile Systems Center, USA, ‘The Aerospace Corporation, USA, *University of Texas at El Paso - 
Jacobs JETS Contract, NASA Johnson Space Center, Houston, TX, USA, ‘Jacobs, NASA Johnson Space Center, 
Houston, TX, USA. 


To improve prediction accuracy, the DebriSat project was conceived by NASA and DoD to update existing standard 
break-up models. Updating standard break-up models require detailed fragment characteristics such as physical size, 
material properties, bulk density, and ballistic coefficient. For the DebriSat project, a representative modern LEO 
spacecraft was developed and subjected to a laboratory hypervelocity impact test and all generated fragments with at 
least one dimension greater than 2 mm are collected, characterized and archived. Since the beginning of the 
characterization phase of the DebriSat project, over 130,000 fragments have been collected and approximately 250,000 
fragments are expected to be collected in total, a three-fold increase over the 85,000 fragments predicted by the current 
break-up model. The challenge throughout the project has been to ensure the integrity and accuracy of the 
characteristics of each fragment. To this end, the post hypervelocity-impact test activities, which include fragment 
collection, extraction, and characterization, have been designed to minimize handling of the fragments. The procedures 
for fragment collection, extraction, and characterization were painstakingly designed and implemented to maintain the 
post-impact state of the fragments, thus ensuring the integrity and accuracy of the characterization data. Each process 
is designed to expedite the accumulation of data, however, the need for speed is restrained by the need to protect the 
fragments. Methods to expedite the process such as parallel processing have been explored and implemented while 
continuing to maintain the highest integrity and value of the data. To minimize fragment handling, automated systems 
have been developed and implemented. Errors due to human inputs are also minimized by the use of these automated 
systems. 

This paper discusses the processes and challenges involved in the collection, extraction, and characterization of the 
fragments as well as the time required to complete the processes. The objective is to provide the orbital debris 
community an understanding of the scale of the effort required to generate and archive high quality data and metadata 
for each debris fragment 2 mm or larger generated by the DebriSat project. 


I, BACKGROUND AND CURRENT STATUS 


In order to characterize the DebriSat fragments, a 


The DebriSat test article is a representative of a 
typical modern low Earth orbit (LEO) satellite that was 
subjected to a laboratory hypervelocity impact (HVJ) test 
in April 2014 [1,2]. The overall objective of DebriSat 
project is to update existing standard satellite break-up 
models with components and materials common to 
modern LEO spacecraft. For the laboratory HVI test, the 
DebriSat test article was suspended within a “soft-catch” 
arena formed by polyurethane foam panels to protect the 
debris fragments from the metal walls of the test 
chamber. After the laboratory HVI test, all the foam 
panels and debris fragments within the test chamber were 
collected and transported to University of Florida for 
fragment processing and characterization. Since the 2014 
laboratory HVI test, the DebriSat team has been 
collecting, extracting, characterizing, and archiving 
fragments down to 2 mm in length. 
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rigorous process was developed and implemented. There 
are three major tasks associated with the post-impact 
processing of DebriSat fragments: detection, extraction, 
and characterization. Each task and its associated sub- 
tasks and activities are shown in Figure |. The detection 
task has three sub-tasks beginning with foam panel 
preparation where loose fragments and fragments 
embedded on the surfaces of the panels are collected and 
stored. Once panel preparation is completed, the panels 
are X-ray imaged. After the X-ray images are acquired, 
they are post-processed to detect other embedded 
fragments. The extraction task involves the careful 
extraction of fragments with at least one dimension 
greater than 2 mm which are stored individually. The 
approximate location of each fragments on or in the panel 
is recorded to provide information on_ physical 
distribution and penetration depth. The fragments are 
recorded into the DebriSat Categorization System (DCS) 


Page 1 of 12 


68" International Astronautical Congress 2017, Adelaide, Australia. 


database where each fragment is given a unique 
identification number [3,4]. The characterization task 
involves fragment assessments (2D/3D _ size 
discriminator, material, shape, and color), measurements 
(mass and physical size Xpm, Ypm, and Zpm), and the 
size characteristics from the measurements (_e., 
determining the characteristic length, average cross- 
sectional area, and volume) are calculated. All the 
fragment data and images generated throughout the 
process as well as the associated metadata (revision 
history, creator, timestamp, etc.) are all uploaded and 
stored on the DCS. Once the fragments are characterized, 
each fragment goes through a verification process where 
technicians confirm the data archived in the DCS 
database. The fragment verification ensures the integrity 
of the fragment data generated throughout the post-HVI 
process. In addition, Gage repeatability and 
reproducibility (Gage R&R) tests are conducted every 
1,000 verified fragments to quantify the variations in 
equipment and procedure. The Gage R&R tests maintain 
high levels of confidence in the fragment characterization 
data. 
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Figure | Post-HVI processes 
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A total of 588 foam panels were installed inside the 
test chamber during DebriSat’s laboratory HVI test. 
Three different density panels, denoted low, medium, and 
high, were stacked and utilized to capture debris 
fragments inside the chamber. The foam panels were 
arranged such that the density of the panels increased 
radially out towards the wall of the test chamber. 

Table 1 shows the progression of debris fragment 
processing from the post-HVI test to the present (as of 
August 2017). Panel preparation and recording fragments 
in the database were the main focus in 2014 — 2015. The 
focus for the 2015 — 2016 was to X-ray the foam panels, 
extract fragments from the foam panels, and to begin 
fragment characterization. The recent efforts have been 
focused on fragment extraction and _ fragment 
characterization, while there are steady increases in panel 
preparation and X-ray image acquisition. No fragments 
were verified in the first two years, however, there have 
been close to 3,000 fragments that have been verified 
thus far. 


Table | Status of post-HVI processing 


2014- 2015- 2016- 
2015 2016 2017 
Panels prepared 304 369 382 
Panels X-rayed None 279 295 
and post-processed 
Panels extracted None 62 285 
Fragments 90,000 125,000 154,000 
collected 
Fragments 73,571 117,712 138,210 
recorded in DCS 
Fragments None 9,344 31,144 
extracted 
Fragments None 882 13,886 
characterized 
Fragments verified None None 2,842 


As of August 2017, of the 588 panels used to 
construct the soft-catch arena, 382 full panels have been 
prepared for X-ray imaging; for compactness we denote 
this as 382/588 panels have been processed for X-ray 
imaging. Full panels are defined as panels with greater 
than 2/3 of its planform intact and the remaining 208/588 
panels are broken into pieces smaller than 2/3 of the 
panels’ planform. Embedded objects have been identified 
in 295/382 prepared panels which have been X-ray 
imaged and fragments have been extracted from 282/295 
of the X-ray imaged panels. The processing of the 382 
full panels took over 1,200 hours (i.e., 0.6 person-year) 
to complete, averaging approximately 3 hours per panel. 
The X-ray imaging of 295 panels took 150 hours to 
complete, averaging approximately half an hour per 
panel. To complete extracting fragments from 285 full 
panels took over 2,800 hours (i.e., 1.4 person-year), 
averaging 10 hours per panel. The average times to 
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extract fragments from medium and high density panels 
are less than 6 hours, however, the times for low density 
panels is approximately 22 hours. Since the panels were 
arranged in the test chamber such that the panel density 
increased radially outward, during the impact test 
majority the fragments impacted the low density panels 
and only the more energetic fragments reached the higher 
density panels (shown in Figure 2). 


Table 2 Processing times per panel 


Number Total Average 

of panels time time 

(hr) (hr) 
Panel preparation 382/588 12224 3.2 
X-ray imaging 295/382 150.0 0.5 

Extraction 282/295 2876.4 10.2 

Low density 78 1723.8 22.1 
Medium density 135 810.0 6.0 
High density 69 165.6 24 


oe 


Figure 2 Panel layout inside chamber (left) and the 
aftermath (right) 


An initial estimate of 85,000 debris fragments were 
predicted by the current NASA satellite break-up model. 
To date, 138,210 fragments have been collected and 
entered into the DCS database and approximately 15,790 
fragments are awaiting database entry for approximately 
154,000 fragments that have been collected post-HVI 
testing. Over 107K fragments have been collected and 
recorded from panel preparations and over 31K 
fragments have been extracted and recorded from the 
extraction activities. Out of the 138,210 fragments that 
are archived in the DCS database, 13,886 fragments have 
completed characterization (assessment, mass, and size 
measurements) and of that 13,886 fragments, 2,842 
fragments have completed verification. The fragment 
counts are shown in Table 3. To note, the characterization 
effort began with fragments that qualify as 2D (i.e., small 
and/or negligible thickness) and carbon fiber reinforced 
polymer (CFRP). Details of characterization using the 2D 
imaging system will be discussed in ILI]. As of 2017, 
the effort to characterize fragments using both the 2D and 
3D imaging systems in underway to extract information 
with more complex, sizeable fragments. 

In this paper, the processes and challenges involved 
in the collection, extraction, and characterization of the 
DebriSat fragments are discussed. Each task and 
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activities are rigorously designed, developed, and 
implemented to ensure that all fragment data and 
associated metadata are properly captured and archived 
in the database. Due to the sheer number of fragments 
and their associated data to be captured and stored, as 
well as the ever changing group of technicians 
(undergraduate students), the post-processing of 
DebriSat fragments is a daunting task. However, to 
address these challenges, many of the processes have 
been automated and streamlined to increase fragment 
processing throughput as well as ensure independence of 
the technicians (i.e., minimize subjectivity) on the 
generated data. The processing facility layout continues 
to be updated to organize activities and workstations such 
that the post-HVI activities are streamlined. The paper is 
outlined as follows: details the post-HVI processes, 
challenges and updates implemented throughout the post- 
processing activities to efficiently yet at high quality, 
collect and archive the DebriSat fragment data. 


Table 3 Number of fragments recorded in the DCS 
Number of fragments 


Panel preparation 107,066 
Extraction 31,144 
Characterization 13,886 
Verification 2,849 
Total in the DCS 138,210 


Il.POST-HVI PROCESSES AND UPDATES 
The post-HVI process is decomposed into three major 
tasks: detection, extraction and characterization. In this 
section, each task is discussed to highlight the challenges 
and the resolutions that were implemented. 


II.I Detection 

The soft catch arena installed in the chamber was 
constructed with 588 panels organized into three 
sections: up-range where the projectile entered the arena, 
side which protects the fragments from impacting the 
chamber walls, and down-range which acts as a backstop 
to prevent fragments from blowing downstream of the 
impact point. The up-range section had 24 panels 
installed, the side section had 452 panels, and the down- 
range section 112 panels. The side section was further 
subdivided into five rows with Row | closest to down- 
range section and Row 5 closest to up-range section. 
Further details of the panel distribution are shown in 
Table 4. Each panel was designated based on their 
chamber location and their placement in the panel stack, 
which was used to uniquely identify and archive the 
information in the DCS database. To date, 382 panels 
have been processed and prepared for X-ray image 
acquisition. These 382 panels are panels that are at least 
2/3 of the original panel planform (referred to as full 
panels) and the remaining panels are all less than the 2/3 
of the original panel planform (referred to as broken 
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panels). A procedure to prepare the full panels have been 
developed and implemented, and a procedure for 
preparing broken panels is developed and is ready to be 
implemented. The procedure for preparing broken panels 
is different from the procedure for full panels due to its 
smaller form. To provide all broken panels with unique 
identification would be prohibitively time consuming, 
therefore, unique foam panel IDs are only given to 
broken panels with a minimum length greater than 10 cm. 


Table 4 Foam panels prepared 


Chamber section Panels in Panels 
chamber prepared 

Down range 112 43 

Up range 24 18 

Side 452 319 
Row | 92 84 
Row 2 92 85 
Row 3 92 76 
Row 4 92 30 
Row 5 84 44 
Unknown 2 

Total panel count 588 382 


Of the prepared full panels, 109/382 were low 
density, 205/382 were medium density, and 66/382 were 
high density. Two of the panels were severely charred 
and their designations were not visible, thus, their row 
identifications are denoted as unknown in Table 4. 

The preparation times for each panel density category 
are shown in Figure 3 and Table 5, specifically, the 
average, maximum, and minimum times required to 
prepare the panels are shown. The preparation times for 
the low density panels are significantly greater than the 
preparation times for medium and high density panels, 
since most of the fragments were captured by the low 
density panels. 
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Figure 3 Full foam panel preparation times 
Once the full panels were prepared, they were 


transported to the X-ray facility and imaged. Due to the 
size limitation of the X-ray detector, 12 images are 
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required to cover the entire full panel. The X-ray images 
are then post-processed where they are stitched together 
and an object detection algorithm applied to detect 
embedded fragments. To date, a total of 295 full panels 
have been X-ray imaged and post-processed. The average 
X-ray image acquisition time is half an hour per panel 
(see Table 1) and a few minutes to post-process each set 
of panel X-rays. A malfunction of the X-ray equipment 
has caused a delay in the imaging of the remainder of the 
prepared panels. Once the X-ray imager is repaired and 
functional, the team will complete X-ray imaging of all 
prepared panels. X-ray imaging of broken panels will 
commence once all full panels have been X-ray imaged. 
However, rather than imaging each broken panel 
individually, multiple broken panels will be made into a 
faux panel and imaged and post-processed. A procedure 
to image and post-process the broken panels is currently 
being developed and will be verified prior to 
implementation. The X-ray image acquisition and post- 
processing times are to be analyzed once the broken 
panel X-ray imaging begins. 


Table 5 Foam panels preparation times 


Preparation time Low Medium High 
Panel count 109 205 66 
Max (hrs) 94.2 20.4 6.2 
Average (hrs) 12 1.4 1.0 
Min (hrs) 0.5 0.2 0.2 


ILI Extraction 

Prior to carefully extracting the fragments identified 
from the post-processed X-ray images, each panel is 
verified to ensure that the preparation data are properly 
entered into the DCS database. With the panels verified, 
fragments with at least one dimension greater than 2 mm 
are carefully extracted and archived in the DCS with 
unique fragment identification numbers. The extraction 
procedure is to scrape away at the surface of the foam 
panels with tweezers and carving tools to excavate the 
debris fragments. The procedures differ slightly 
depending on the density of the foam panels, however, 
the main difference is merely the choice of tools used to 
excavate the fragments from the foam panels [1]. 

Currently, a total of 285 full panels have completed 
extraction, out of which 79 panels are low density, 135 
panels are medium density, and 71 panels are high 
density. The average, maximum, and minimum 
extraction times for each panel density are shown in 
Figure 4 and Table 6. Similar to the preparation times, 
the low density panels require more time for extraction 
on average due to the higher fragment counts captured in 
them. There was a low density panel that took over 160 
hours to complete extraction and over 3,500 fragments 
were collected from that panel. For some high density 
and medium density panels, there were no embedded 
fragments identified through the X-ray images, therefore, 
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the surfaces of the panels were carefully inspected to 
ensure that no fragments are captured in the panels. It 
should be noted that, the extraction task involves two to 
four technicians per panel. 


=Max Average =Min 
180 
© 160 
= 140 + 
£ 120 | 
F 100 
§ 80 | 
o 60 
§ 40 
ui 20 T 
0 2 
Low Medium High 
Panel Density 


Figure 4 Foam panel extraction times 


Table 6 Foam panels extraction times 


Extraction time Low Medium _High 
Panel count 719 135 71 
Max (hrs) 161.3 38.8 24.3 
Average (hrs) 21.9 6.0 2.3 
Min (hrs) 0.8 0.2 0.1 


After extraction was completed on several panels, 
they were re-X-ray imaged to ensure no fragments were 
missed during extraction. No fragments were identified 
in the reimaged medium and high density panels; 
however, several fragments were detected in some 
reimaged low density panels. One reason why there were 
some fragments detected from the post-extracted low 
density panels was that those panels were processed with 
an early version of the object detection algorithm. The 
early object detection algorithm had challenges in 
stitching the 12 X-ray images, where the fragments 
located in the boundaries of the images were not 
identified. In addition, the early algorithm filtered objects 
smaller than two pixels, thus, the smaller fragments were 
not properly identified. However, updates have been 
made to the object detection algorithm where the image 
stitching has improved to detect fragments in the 
boundaries and the two-pixel filter has been removed. 
The extracted low density panels are re-extracted to 
recover fragments that were missed due to the early 
object detection algorithm. It is anticipated that once 
panels that were initially processed using the updated 
object detection algorithm are reimaged this will no 
longer be an issue. However, to date, only those panels 
that were processed with the early version of the 
detection algorithm have been reimaged. 
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Il. 1M Characterization 

All fragments collected during preparation and 
extraction are individually bagged, assigned a unique 
fragment identification number, and entered into the DCS 
database. First, the panel information associated with the 
fragment are entered and this information provides 
general insights into the location of the fragment post- 
HVI. Next, the fragment’s physical attributes are 
assessed and entered; the fragment’s physical attributes 
are (i) size category (i.e., 2D or 3D fragment), (ii) 
material property, (iii) shape, and (iv) color. Once all 
assessments are completed, the fragment’s mass and size 
are measured and entered into the DCS database along 
with associated metadata. Prior to finalizing the database 
entries associated with a fragment, an independent 
technician (one who has no prior involvement with the 
fragment) reviews and certifies that the database entries 
are valid; once this certification has been completed, the 
fragment database entry is locked and no further 
modifications are possible. 

The initial characterization effort was focused on 
fragments that were categorized as 2D and these were 
primarily carbon fiber reinforced polymer (CFRP) 
fragments. This focus was based on the fact that the first 
size characterization system to become available was the 
2D imaging system. The current characterization process, 
however, has been expanded to include characterization 
of non-CFRP fragments and shortly will begin 
characterization of fragments categorized as 3D. 

The assessment, measurement, and the verification 
processes have been updated and are discussed in detail. 


Assessment 

After the fragment’s associated panel information are 
entered into the DCS, the fragment’s size category, 
material property, shape, and color are assessed. The 
fragments are categorized as 2D or 3D depending on 
which imaging system will be used to measure the 
fragment’s physical size parameters (Xpm, Yom, Zp, 
average cross-sectional area, and volume). Initially the 
categorization of a fragment as 2D and 3D was based on 
error analysis [2], however, with the improvements to the 
2D imaging system and a detailed characterization of the 
3D imaging systems, the size threshold was determined 
based on the performances of the imaging systems. The 
size threshold has been set at 3 mm; i-e., all fragments 
with a minimum dimension of 3 mm or greater are 
measured on the 3D imaging system and all other 
fragments will be measured on the 2D imaging system. 

Once the fragments are organized into their respective 
size category (2D or 3D), their material property, shape, 
and color are assessed. The fragments are placed under a 
microscope such that the technicians can better inspect 
the fragment features. There are 13 material categories 
based on the materials used in the fabrication of 
DebriSat: aluminum, carbon fiber reinforced polymer 
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(CFRP), copper, epoxy, glass, Kapton tape, Kevlar, 
multi-layered insulation, printed circuit board (PCB), 
plastics, solar cells, silicone, stainless steel, and titanium. 
In addition to this list, a “metal” category was added to 
aid the technicians during the material assessment. There 
are three materials that fall under this “metal” category: 
aluminum, stainless steel, and titanium, due to their 
similar appearances. When the “metal” is selected, the 
material is later identified based on the density 
calculation from the mass and size measurements (this is 
one of the items the independent verifier examines during 
the certification process). To aid in the assessment of the 
fragment’s material property, the fragments are 
compared to samples and pictures of each material type. 

The number of shape categories remain the same at 
six, however, improvements were made in the definitions 
of each shape to help the technicians. The six shape 
categories are: (i) straight rod/needle/cylinder, (ii) bent 
rod/needle/cylinder, (iii) flat plate, (iv) bent plate, (v) 
nugget/parallelepiped/spheroid, and (vi) flexible. Several 
samples of each shape category are provided to the 
technicians to assist in the fragment shape assessment. 

The color categories have been reduced by one to 13. 
Previously, the color category included a “burnt/charred” 
option, however, it has now been removed. The list and 
definitions of each shape and color category are 
explained in Ref. [2]. Once the assessments are 
completed on the fragments, the fragments are passed 
onto mass and size measurements. 

As of the writing of this article (August 2017), there 
is a total of 19,314 fragments (all 2D) that have 
completed assessment. Out of these fragments, 17,098 
fragments are assessed as CFRP, and 2,216 fragments are 
assessed as non-CFRP material. 


Mass Measurement 

Once the fragment assessments have been completed, 
the next step in the fragment characterization process is 
the measurement of its mass. Three mass balances that 
have been selected to perform the mass measurements: a 
BM-22, a PGL-203, and a CY-510. The three balances 
were selected based on their readability, sensitivity, and 
maximum capacity as shown in Table 7. The BM-22 is a 
microbalance that meets the measurement requirements 
and is capable of measuring DebriSat fragments that are 
very small. Due to its high sensitivity to the slightest 
disturbances, the microbalance was placed under a draft 
shroud on a granite table. 


Table 7 Summary of mass balances used for mass 
measurement 


As the fragment mass measurements continued, it 
was determined that another microbalance was needed to 
increase the productivity. As a result, a second 
microbalance was acquired and incorporated to measure 
the fragment masses. Similar to the first microbalance, 
the second microbalance is placed under a draft shroud 
on a separate granite table. To date, approximately 
17,000 fragments have been massed on one of the two 
microbalances (see Figure 5). The breakdown of the 
fragments that were massed on each microbalance are 
shown in Table 8, where the first and second 
microbalances are denoted as Micro 1 and Micro 2, 
respectively. As seen in Figure 5, the majority of the 
fragments that have been massed so far have very small 
mass. Table 9 shows that the average mass of the 
fragments massed to date is less than 250 milligrams. As 
the characterization continues, bigger fragments will be 
massed and the maximum and average mass 
measurements will change significantly from what is 
shown in Table 9. 
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Measurements 
Figure 5 DebriSat fragment mass measurements as of 
August 2017 


Table 8 Mass measurements as of August 2017 
Fragment count 


Micro 1 11,686 
Micro 2 5,243 
Total massed 16,929 


Table 9 Mass measurements of 17K fragments as of 


August 2017 
Parameters Masses (g) 
Maximum 4.292788 
Average 0.00243 1 
Minimum 0.000001 


Capacity Readability 


Model (g) (g) Std. Dev. (g) 
BM-22 5 0.000001 0.000004 
PGL 203 200 0.001 0.002 
CY-510 510 0.001 0.001 
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Due to the sensitivity of the microbalance to 
environmental effects, the temperature and the humidity 
at the time of each mass measurement is also recorded 
and archived. A temperature and humidity sensor was 
placed within 2 m of each microbalance so that the 
temperature and humidity are also measured at the time 
of each mass measurement. The temperature and 
humidity measurements from the approximately 17,000 
mass measurements are shown in Figure 6 and Figure 7, 
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respectively and a summary of that data are shown in 
Table 10. Both the temperature and humidity 
measurements are well within the acceptable working 
conditions of the balances. As the characterization efforts 
continue, larger fragments will be massed and will have 
their masses measured with the other two balances. 


Table 10 Temperature and humidity measurements of 


17K fragments 
Parameters Max Average Min 
Temperature (C) 29.6 24.1 20.3 
Humidity (%) 59.4 44.5 23.2 
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Figure 6 Temperature measurements at each mass 
measurement 
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Figure 7 Humidity measurements at each mass 


measurement 


Size Measurements 

The next step in the characterization process is the 
determination of the fragment’s size characteristics (Lc, 
average cross-sectional area, and volume). Based on the 
fragment’s size categorization as either 2D or 3D, the 
fragments size characteristics are measured using either 
the 2D imaging system or the 3D imaging system. Both 
imaging systems utilize point-and-shoot cameras for 
fragment image acquisition>. The images are then 
processed to generate their fragment’s size 
characteristics. The fragment size characteristics, 
images, and associated metadata (e.g., revision history, 
timestamps, etc.) generated during the measurement 
process are archived in the database. 

The initial configuration of the 2D imaging system 
assumed negligible height (i.e., Zpmm=0 since the CFRP 
fragments have negligible height) and only the 
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fragment’s two largest orthogonal in-plane dimensions 
(denoted as Xpm and Ypm) were measured. However, as 
the characterization activities progressed and non-CFRP 
fragments were characterized, a right angle wedge mirror 
was incorporated to capture the height of the fragment. 
The in-plane dimensions Xprm and Ypimw are measured by 
processing the back light image (i.e., silhouette) and the 
height, Zprm, is obtained from the front lit image (see 
Figure 8). 


Back light image 
Figure 8 Back lit and front lit images from updated 2D 
imaging system with mirror 


Front light image 


There are two 2D imaging systems that have been 
developed and implemented to increase the productivity 
during fragment characterization process. To ensure that 
the characteristic length error requirement was being 
satisfied with the inclusion of the mirror to measure the 
third longest dimension, Zp, two hollow disks were 
used to examine the characteristic length accuracies of 
both 2D imagers. The hollow disks have the same outer 
diameter and inner diameter and their only difference is 
the heights, 0.25 mm and 0.13 mm, respectively. Ten 
measurements were taken for each hollow disk and on 
each imager. The characteristic length (Lc) error was 
calculated by comparing the 2D imaging system 
measurements to the actual dimensions of the hollow 
disks. The hollow disk dimensions were physically 
measured using micrometers and their characteristic 
lengths were calculated. Table 11 shows the Lc 
accuracies of both hollow disks on the two 2D imaging 
systems with the mirror and without the mirror. The Lc 
accuracies are well within the +10% requirement with 
and without the mirror, but the accuracies improve with 
the addition of the mirror. Therefore, all pre-mirror size 
measurements are acceptable and do not need to be 
repeated. Designations have been added to the database 
to identify pre- and post-mirror size characterized 
fragments. 


Table 11 Lc accuracy of the two 2D imagers 


Imager Disk Error without Error with 
(mm) mirror (%) mirror (%) 

1 0.25 -1.93 -1.22 

1 0.13 -1.18 -0.90 

2 0.25 -1.18 -0.55 

2 0.13 -0.54 -0.26 
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To date, 13,885 2D fragments have completed 
assessment, mass, and size measurements. Out of these 
fragments, a little over 9,000 fragments had their sizes 
measured prior to the addition of mirror and the 
remaining fragments had their sizes with measured with 
the mirror setup. The distributions are shown in Table 12. 


Table 12 Size measurements (2D) as of August 2017 
Fragment count 


Pre-mirror setup 9,346 
2D Imager | 2,547 
2D Imager 2 1,992 
Total characterized 13,885 


When fragments are categorized as 3D, their sizes are 
measured on the 3D imaging system. The 3D imaging 
system consists of a six-camera configuration with a 
turntable and studio lighting (to minimize shadowing). 
The 3D object on the turntable is reconstructed from 
multiple 2D images using a space-carving technique 
[6,7]. From the space-carved object, the representative 
3D point cloud is generated and the characteristic length 
and volume are calculated. The average cross-sectional 
area is calculated as the average of the projected areas, 
where the projected areas are calculated based on the 
object silhouettes [8]. The fragment measurement data 
and their images are all archived in the DCS database. 
Currently, the performance of the 3D imaging system is 
being verified and once completed the system will be put 
in service. To date, no fragment size has been measured 
on the 3D imaging system. 

To verify the performance of 3D imaging system, 
four shapes (a cone, a rectangular prism, a square 
pyramid, and a sphere) and three sizes for each shape 
(small, medium, and large) were characterized. The 
objects are shown in Figure 9 and their dimensions are 
provided in Table 13. 


Figure 9 Objects used to characterize 3D imaging system 


For each object, ten measurements were performed 
on the 3D imaging system. The Lc errors were calculated 
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for each object by comparing the measured/computed 
values to their actual values. Table 14 shows that the Lc 
errors for each object are within the requirement. 


Table 13 Object dimensions (in mm) 


Shape Large Medium Small 

Prism 12.57 x 9.56x9.56 9.59 x 6.22 
12.73 x x 25.39 x 25.40 
25.31 

Cone 29.97 dia.x 19.92 dia.x 10.00 dia. x 
29.76 19.74 9.71 height 
height height 

Pyramid 15.22x 10.15 x 4.99 x 4.97 
15.23 x 10.11 x x 9.38 
29.31 19.33 

Sphere 30.44x 20.31 x 10.78 x 
30.15 x 20.10 x 10.03 x 
29.80 19.91 9.96 

Table 14 Lc errors with 101 images 

Large Medium Small 

Cone -1.42% -1.32% -2.09% 

Prism -3.01% -3.34% -4.55% 

Pyramid -4.51% -4.10% -8.68% 

Sphere -1.44% -2.35% -2.69% 

Verification 


The final step in the characterization process and 
perhaps the most important is verification. The 
verification step involves a pass or fail logic to analyze 
and confirm all the appropriate data for each fragment are 
stored in the DCS database. Verifying the debris 
fragments requires careful attention to detail and 
knowledge of the entire DebriSat characterization 
process. The technicians analyze the fragment data 
starting with the panel identification and chamber 
location and checks if the corresponding panel 
information is correct. The next step involves checking 
the fragment’s material, shape, and color under a 
microscope and compares them to the information in the 
database. Next, the mass measurement and _ size 
measurements in database are compared to the fragment. 
If there are any inconsistencies in the measurements, the 
verification fails. The final step is to check that all images 
and the point cloud files are present and correct. If any of 
the images are absent or incorrect, the fragment fails 
verification and must be re-characterized. To ensure 
accurate results, the verifying technician cannot 
previously be associated with the fragment. Once the 
fragments complete verification no further modifications 
are possible. 

During each characterization activity, the processing 
times were examined. Table 15 shows the processing 
times for assessment, mass and size (2D) measurements, 
and verification; specifically, the average time per 
fragment is determined based on the number of fragments 
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examined. The number of fragments that were used to 
calculate the average time per fragment differs from the 
total number of processed fragments since the time 
information was not captured for all the processed 
fragments. On average, the assessment (2D/3D, material, 
shape, and color) takes around 5.5 minutes per fragment, 
which includes the setup as well (i.e., removing the 
fragment from its storage bag and placing it under the 
microscope). For mass measurement, it takes around 6.3 
minutes per fragment to measure the mass, measure the 
temperature and humidity, and to archive the data in the 
DCS database. This average time includes the initial 
waiting period for the microbalances to adapt to the 
environment as well as weighing the fragments. The 
average time to measure the fragment size on the 2D 
imaging system is around 7.1 minutes. This time includes 
the setup of the equipment as well as the measurement 
and uploading to the database. The average time for the 
size measurement is higher than the assessment and mass 
measurements since the data size uploaded to the DCS 
from the 2D imaging system is much larger due to 
images. The average time to verify the fragment data in 
the DCS database is around 6.2 minutes. The average 
times per fragment will continue to be examined to 
identify where improvements can be made to increase 
productivity. 


Table 15 Average characterization time per fragment 
Average time 


Fragment 
per fragment 
count P 
(minutes) 
Assessment 2,694 5.5 
Mass measurement 16,929 6.3 
Size measurement (2D) 8,298 7A 
Verification 743 6.2 


To date, a total of 2,842 fragments have been verified 
and two Gage R&R tests [9,10,11] have been conducted 
to assess the repeatability and reproducibility of the 
processes and devices used in the characterization 
process. The results from the Gage R&R tests are 
discussed in Section HI. 


Il.GAGE REPEATABILITY AND 
REPRODUCBILITY TEST 

Gage repeatability and reproducibility (Gage R&R) 
test is a tool commonly used to quantify the amount of 
variation in a measurement system and the sources of the 
variations [9,10,11]. Repeatability measures the 
contribution to the gage variance when the same part is 
measured multiple times with all other factors held 
constant. Reproducibility gives contribution to the gage 
variance from additional factors, in this case, the 
operators. In order to deem a measurement system 
acceptable, the total variation must be below 10% and for 
conditional acceptance below 30%. The measurement 
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system is unacceptable when the total variation is greater 
than 30%. 

The components that effect variability are 
repeatability and reproducibility. Repeatability is an 
indicator for variability from the equipment; in the 
current case this includes both microbalances and the 2D 
imaging systems. Reproducibility is an indicator to 
whether or not the characterization process and 
procedure carried out by technicians are good. The total 
Gage R&R is computed as the root-mean-square of the 
repeatability and reproducibility. The analysis of these 
varying factors is important for showing the procedures 
are developed in a way that can be utilized by any 
operator. As such, the Gage R&R tests are conducted for 
every 1,000 fragments that pass verification and to ensure 
integrity of data by a random selection of five fragments 
and three experienced technicians. The study utilizes 
Minitab software [10] to compare measurements taken 
during the testing to the original database entry. 

To carry out the Gage R&R test, three technicians 
with characterization experience were randomly 
selected. The test started with both micro mass balances 
and concluded with the 2D imagers. The assessments 
were not included in the study because the assessment 
data are qualitative with no equipment used. First, each 
of the technicians took the mass of five CFRP flat plate 
fragments at random with each mass balance. For each 
fragment, the mass measurements taken by the 
technicians and the mass measurement stored in the DCS 
database were analyzed. Next, the same five fragments 
were imaged by the same three technicians on both 2D 
imaging systems to measure the fragments sizes. For 
each fragment, the size measurements taken by the 
technicians and the size measurement stored in the DCS 
database were analyzed. All the data from mass 
measurements and size measurements were analyzed 
using Minitab’s expanded Gage R&R tool. 


Il.I Gage R&R Test 1 

The first test was conducted after the first 1,000 
DebriSat fragments were verified and the second test 
after the second 1,000 DebriSat fragments were verified. 
For both tests, five CFRP flat plates were selected at 
random and the masses and sizes were measured. CFRP 
flat plate fragments were chosen instead of needle-like 
simply because they are sturdier and would survive the 
significant amount of handling during the test. 


Mass Measurement Results 

The first Gage R&R test took place over a period of 
two days due to ensuring the process was correct. All of 
the mass measurements were taken during the first day 
and the results are shown in Table 16, Figures 9 and 10. 
Table 16 and Figure 10 shows the variations that were 
calculated based on the mass measurements and Figure 
10 shows the variations between technicians (including 
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the data from DCS denoted as Technician 4) for each 
fragment. The total Gage R&R was 7.3%, which is in the 
acceptable range for the test criteria. The variations are 
coming from the equipment rather than the technicians 
and the procedure. The part-to-part variation examines 
the differences between the test articles and a high 
variation suggests that each gage is able to distinguish 
each part that is tested is different. Figure 11 shows that 
the variations in mass measurements of each fragment by 
the technicians are very low and that the mass 
measurement processes are very good. 


Table 16 Test 1: Calculated mass measurement variations 
Mass variation (%) 


very good. The variations are mainly from the equipment 
and not the technicians. The Gage R&R for the Xpm and 
Ypm are very low with 11.25% and 5.77%, respectively, 
however, the Gage R&R for the Zpm is 54.46% which 
surpasses the acceptable range and is in the unacceptable 
range. An explanation for the significant Gage R&R for 
the Zp is the Zpm measurements taken by the 
technicians during the tests were determined using the 
mirror, however, the Zpm data from the DCS was 
calculated and derived based on mass and density. In fact, 
this large Gage R&R value is a positive result because it 
detected the process change. 


Table 17 Test 1: Calculated size measurement variations 


Repeatability 7.30 
Reproducibility 0.00 
Total Gage R&R 7.30 
Components of Variation 
100 [i * Contribution 
[By % Study Var 
Bs 


o.—___ —_ —_ 
Gage R&R Repeat Reprod Part-to-Part 


Figure 10 Test 1: Components of variation mass 
measurements 


Test Fragment * Technician Interaction 


0.0024 
I] 
% 0.0016 
$ 
~ 0.0008 
i 2 3 4 5 
Test Fragment 


Figure 11 Test 1: Mass measurement variations between 
technicians for each fragment 


Size Measurement Results 

The size measurements of the five fragments were 
taken on the second day of the first Gage R&R test. The 
measurements were taken on both 2D imaging systems 
and the results were analyzed with the size data in the 
DCS database. There are three dimensions associated 
with each size measurement, Xp, Yom, and Zpm 
corresponding to the three largest orthogonal dimensions, 
therefore, the Gage R&R were analyzed for each 
dimension. Table 17 shows the variations for each 
dimension Xpmu, Ypm, and Zp, respectively. Similar to 
the results from the mass measurements, the variations 
due to reproducibility is low, indicating that the 
procedures developed for the measurements are both 
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Variations eta vom fais 
(%) (%) (%) 
Repeatability 9.01 5.56 36.26 
Reproducibility 6.73 1.54 40.64 
Total Gage R&R 11.25 5.77 54.46 


Gage R&R Test 1 Results 

The Gage R&R results from both the mass and size 
measurement tests showed that the measurement systems 
and the processes are acceptable. However, the 
measurements were averaged between the two balances 
and two 2D imaging systems rather than examining the 
variations for each equipment independently. Therefore, 
the procedure was updated to examine the equipment 
variation individually. These independent tests were 
performed in the second Gage R&R test. 


Il. Gage R&R Test 2 

The second test was conducted after the second 1,000 
DebriSat fragments were verified. The procedures for 
performing the Gage R&R tests were updated to examine 
the equipment variation individually rather than the two 
microbalances as one equipment and the two 2D imaging 
systems as one equipment. 

Mass Measurement Results 

Three technicians measured masses of five fragments 
in random order for each microbalance. The mass 
measurements were compared to those on the DCS 
database and the Gage R&R were examined. Table 18 
shows the Gage R&R analyses for microbalance 1 and 
microbalance 2. Both Gage R&R are very low with 
7.28% for microbalance 1 and 5.25% for microbalance 2. 
In addition, the variations are due to repeatability, or 
equipment, and not by the reproducibility, or technicians 
and procedure. 


Table 18 Test 2: Calculated mass measurement variations 
Mass micro 1 Mass micro 2 


Variations 


(%) (%) 
Repeatability 7.28 5.25 
Reproducibility 0.00 0.00 
Total Gage R&R 7.28 5.25 
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Size Measurement Results 

Next, the same five fragments were moved to the 2D 
imaging systems and their sizes were measured once on 
each system by three technicians. The measurements and 
the measurements from the DCS were analyzed to 
calculate the Gage R&R. The results from both 2D 
imaging systems are shown in Table 19 and Table 20, 
where Table 19 shows the variations of 2D imager | and 
Table 20 shows variations of 2D imager 2. The Gage 
R&R for Xpm on both 2D imagers were in the acceptable 
range, however, the variations of Ypm on both 2D 
imagers were concerning. Further investigations are 
ongoing to identify the source(s) of the variations. The 
variations in Zpim are high since two calculations of Zpm 
are compared, one calculation is with the height 
determined from the mirror image and the other 
calculation was derived based on the fragment’s mass 
and density. 


Table 19 Test 2: Calculated size measurement variations 
2D imager | 


Variations Xpm Yom Zp 
(%) (%) (%) 
Repeatability 13.00 16.60 61.74 
Reproducibility 0.00 13.76 59.72 
Total Gage R&R 13.00 21.56 85.90 


Table 20 Test 2: Calculated size measurement variations 
2D imager 2 


Variations Xpm Yom Zp 
(%) (%) (%) 
Repeatability 9.67 17.44 76.73 
Reproducibility 6.44 13.89 36.34 
Total Gage R&R ‘111.62 22.30 84.90 
Gage R&R Test 2 Results 


After 2,000 DebriSat fragments were verified, the 
second Gage R&R test was conducted to examine the 
equipment and procedure variabilities. The second test 
was conducted and analyzed such that the microbalances 
and the 2D imaging systems were examined individually 
rather than treating the two microbalances as one piece 
of equipment and the two 2D imaging systems also as one 
piece of equipment. Similar to the first test, the variations 
from the mass measurements on both microbalances are 
low, indicating that the equipment as well as the 
procedures are very good. In contrast, the size 
measurements on 2D imaging systems were of concern, 
specifically, the variations in Ypm were much higher 
than the variations in Xpm (on both 2D imagers). While 
the Ypm measurements are dependent on Xpm, the 
source of the variation was not conclusively determined. 
Further investigations are ongoing to identify the root 
cause(s) of the variations to determine if the equipment 
and/or the procedure needs to be updated and improved. 
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IV.CONCLUSION AND FUTURE 

Rigorous procedures for each activity in the post- 
impact phase of the DebriSat project has matured to 
improve efficiencies as well as to incorporate new 
processes/equipment based on _ lessons learned. 
Furthermore, the procedures have been developed and 
updated to be independent of technicians to adapt to 
growing and ever changing group of technicians. One of 
the biggest challenge is handling and working with 
fragments in the 2 mm range, and the technicians have 
focused on extreme care and attention during each 
activity. A verification step has been implemented in 
order to mitigate inaccuracies in the information stored in 
the database and to ensure the integrity of the data 
archived for each fragment. 

There has been a steady increase in the panel 
preparation for X-ray imaging and object detection and 
completed all the panels that are greater than 2/3 of the 
original size. The remaining foam panels are all broken 
panels, thus, a procedure has been developed and is 
currently being verified to process the broken panels. As 
the third year of post-impact processing continues, the 
majority of the focus has been on fragment extraction and 
characterization. Over 20,000 fragments have been 
collected through extraction, and over 13,000 fragments 
have been characterized. In addition, close to 3,000 
fragments have been verified. While only fragments that 
qualify as 2D are characterized to date, the 3D fragments 
are beginning to be characterized. 
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